1. Field of the Invention
This invention pertains generally to image capture, and more particularly to a method and apparatus for determining focus direction in a series of images.
2. Description of Related Art
One critical quality for capturing a desired image is that it be properly focused. Numerous systems have been developed for estimating or attaining a proper camera focus. As a camera-lens system has a number of related elements and characteristics, a brief discussion follows of these elements and their associated characteristics.
Generally, the two main optical parameters of a photographic lens are maximum aperture and focal length. The focal length determines the angle of view, and the size of the image relative to that of the object (subject) for a given distance to the subject (subject-distance). The maximum aperture (f-number, or f-stop) limits the brightness of the image and the fastest shutter speed usable for a given setting (focal length/effective aperture), with a smaller number indicating that more light is provided to the focal plane which typically can be thought of as the face of the image sensor in a simple digital camera.
One form of typical simple lens (technically a lens having a single element) is that of having a single focal length (also referred to as a “prime lens”). In focusing a camera using a single focal length lens, the distance between lens and the focal plane is changed therein altering the focal point of the photographic subject onto that focal plane. Thus, though the single focal length lens has a fixed optical relation and focal length, it is used in the camera to focus on subjects across a focal range span. Consequently, one should not confuse the fixed focal distance of a lens with the range of focal distance obtainable on a camera using that lens, whereby adjusting the position of that lens in relation to the focal plane alters focal distance.
In using a single focal length lens one would adjust aperture to select the amount of light with respect to desired shutter speed, and then adjust focus according to the subject-distance, which is also referred to as the focal distance and then capture an image. Often a macro setting is provided with a different focal length selection, on an otherwise single focal length lens, for taking close-up shots. A telephoto lens provides a very narrow angle of view with high magnification for filling the frame with images from distance objects.
Multi-focal length lenses are usually referred to as “zoom” lenses, because image magnification can be “zoomed”, or “unzoomed” as the case may be. Zoom lenses allow the user to select the amount of magnification of the subject, or put another way, the degree to which the subject fills the frame. It is important to understand that the zoom function of these lenses, or camera-lens systems, is conceptually separate from both the focus control and the aperture control.
Regardless of whether a single-focal length lens or multi-focal length lens is utilized, it is important to properly focus the lens for a given subject-distance. An acceptable range of focus for a given focus setting is referred to as “depth of field” which is a measurement of depth of acceptable sharpness in the object space, or subject space. For example, with a subject distance of fifteen feet, an acceptable range of focus for a high definition camera may be on the order of inches, while optimum focus can require even more precision. It will be appreciated that depth of field increases as the focusing moves from intermediate distances out toward “infinity” (e.g., capturing images of distant mountains, clouds and so forth), which of course at that range has unlimited depth of field.
For a single focal length lens at a given aperture setting there will be a single optimum focus setting for a given distance from camera to the subject (subject-distance). Portions of the subject which are closer or farther than the focal distance of the camera will show up in the captured images subject to some measure of blurring, as depends on many factors that impact depth of field. However, in a multi-focal lens there is an optimum focus point for each lens magnification (lens focal length) obtainable by the lens. To increase practicality, lens makers have significantly reduced the need to refocus in response to zoom settings, however, the necessity for refocusing depends on the specific camera-lens system in use. In addition, the aperture setting can require changing in response to different levels of zoom magnification.
Originally, camera focus could only be determined and corrected in response to operator recognition and manual focus adjustments. However, due to the critical nature of focus on results, focusing aids were readily adopted. More recently, imaging devices often provide the ability to automatically focus on the subject, a function which is generically referred to today as “auto focus”. Focus continues to be a point of intense technical development as each of the many existing auto focus mechanisms are subject to shortcomings and tradeoffs.
Two general types of auto focus (AF) systems exist, active auto focus and passive auto focus. In active auto focus, one or more image sensors is utilized to determine distance to the focal point, or otherwise detect focus external of the image capture lens system. Active AF systems can perform rapid focusing although they will not typically focus through windows, or in other specific applications, since sound waves and infrared light are reflected by the glass and other surfaces. In passive auto focus systems the characteristics of the viewed image are used to detect and set focus.
The majority of high-end SLR cameras currently use through-the-lens optical AF sensors, which for example, may also be utilized as light meters. The focusing ability of these modern AF systems can often be of higher precision than that achieved manually through an ordinary viewfinder.
One form of passive AF utilizes phase detection, such as by dividing the incoming light through a beam splitter into pairs of images and comparing them on an AF sensor. Two optical prisms capture the light rays coming from the opposite sides of the lens and divert it to the AF sensor, creating a simple rangefinder with a base identical to the diameter of the lens. Focus is determined in response to checking for similar light intensity patterns and phase difference calculated to determine if the object is considered in front of the focus or in back of the proper focus position.
In another type of passive AF system, contrast measurements are made within a sensor field through the lens. The system adjusts focus to maximize intensity difference between adjacent pixels which is generally indicative of correct image focus. Thus, focusing is performed until a maximum level of contrast is obtained. This form of focusing is slower than active AF, in particular when operating under dim light, but is a common method utilized in low end imaging devices.
Passive systems are notoriously poor at making focal decisions in low contrast conditions, notably on large single-colored surfaces (solid surface, sky, and so forth) or in low-light conditions. Passive systems are dependent on a certain degree of illumination to the subject (whether natural or otherwise), while active systems may focus correctly even in total darkness when necessary.
Recently, improved auto focusing techniques provide subject-distance estimations and/or focus control under a wide range of conditions by using depth estimation techniques.
The depth from defocus approach relies on estimating depth from two or more pictures taken at different camera settings generating different corresponding amount of defocus blur. Depth can be estimated by computing the relative blur amount between the pictures, as shown in U.S. Pat. No. 7,711,201 which is incorporated herein by reference in its entirety; and U.S. Patent Application Publication Nos. 2007/0189750A1, 2010/0080482A1, and 2010/0194971A1, each of which is incorporated herein by reference in its entirety.
In the two pictures case, relative blur may be computed by matching one of the pictures to increasingly blurred versions of the other picture until an optimal match is achieved. Since it is not known which of the two pictures is closer to the in-focus position, both directions are generally computed and then a best match criterion is used to choose direction.
The performance of the two-picture matching for depth for defocus generally depends on how well the significant features in the two pictures are properly aligned. If the two pictures are not properly aligned or significant mismatch exists, two-picture matching for depth for defocus may give incorrect estimation and direction of focus.
Thus, there is a need to estimate the confidence for direction of depth estimation based on two pictures to allow for determination of the direction of in-focus position as computed by depth from defocus two-picture matching.
The present invention fulfills that need as well as others and overcomes shortcomings of previous camera focus techniques.